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ABSTRACT 

This research report explores the extent to which the 
Early Childhood Environment Rating Scale (ECERS) (Harms and Clifford, 
1980) remains a valid instrument when used to make cross-national 
comparisons of quality, and the advantages and disadvantages of using 
ECERS for this purpose. ECERS was developed to be used by a trained 
observer to rate the quality of early childhood settings. The 
majority of studies using this rating scale have been carried out in 
North America by American investigators, but ECERS is now beginning 
to be used by researchers in other countries. Some researchers have 
expressed reservations about itr appropriateness for early childhood 
settings which are different from the settings for which it was 
developed. In this study, ECERS was used as part of a qualitative 
investigation into nursery provision and practice between 1990 and 
1994 in four countries: the United States (New York City), Spain 
(Barcelona), Italy (Arezzo), and Sweden (Harnosand) . Major findings 
include: large variability in ratings and lower mean scores (4.3) 
achieved by programs in New York; uniformly high ratings of programs 
in Sweden (6.9); and mean scores of 5.4 (Spain) and 6 (Italy) for 
ratings of centers. Problems identified in using ECERS included the 
meaning of particular items on the scale, minimal standards, and 
omission of items and whole dimensions of experience. Advantages of 
using ECERS included the short time span for administering the scr<le 
and its ability to provide comparison measures on a number of 
criteria which child care professionals have agreed are significant. 
Limitations include unstated theoretical assumptions, a failure to 
differentiate among high quality programs, the omission of items that 
assess some aspects of programs, and the need for not only the 
child's experience but also the experience of workers and parents 
(and other stakeholders) to be taken into account when assessing 
quality. (DR) 
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Using the Eariy Childhood Environment Rating Scale as a measure to make cross- 
national evaluations of quality: Advantages and Limitations 

Introduction 

How useful is the Early Childhood Environment Rating Scale (ECERS) (Harms & 
Clififord 1980) as a research instrument in making cross-national comparisons of the quality of 
nurseries? 

This paper will briefly outline the ECERS and discuss the extent to which it can be 
used as a universal evaluative tool to measure quality. The author will re&r to her use of the 
ECERS scale in cross national contexts, (four countries), to comment on its advantages and its 
limitations. 

The Early Childhood Environment Rating Scale (ECERS) is a measure that can be 
used by one observer to make ratings of the quality of nursery settings. It was developed by 
Thelma Harms and Richard Clifford in 1980 at The Frank Porter Graham Child Development 
Center at North Carolina University. 

(With others they have also developed several other instruments for measuring the 
quality of daycare,inctuding the Family day Care Rating Scale (1989) and more recently the 
Infant/Toddler Environment Rating Scale ITERS, (1990 Harms, Ctyer and Cliflford). 

The ECERS is reliable. It has an inter-rater reliability of .884 (.25 classrooms)( Harms 
and Clifford 1980). 

The scale is made up of 37 items, chosen by North American early childhood 
professionals as being indicators of quality. The items cover different aspects of the nurseiy 
enwronment. The different items have been grouped to form seven subscales that have a face 
validity - (a recent factor analytic and correlational analyris of the ECERS carried out by Scarr 
et al. (199 1) has not found any empirical support for the existence of separate subscales, rince 
items appear to be highly correlated ^th each other and with the toUd mean score). These 
are; Personal Care Routines; Fumi^ing and Display for Children; Language-Reasoning 
Experiences; Fine and Gross Motor Activities; Creative Activities; Social Development; Adult 
Needs. 
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There are a varying number of items per subscale ranging from four items to seven. They are 
grouped as follows; 
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Personal Care Routines 


Creative Activities 


Greeting/departing 


Art 


Meals/snacks 


Music/movement 


Nap/rest 


Blocks 


Toileting 


Sand/water 


Personal grooming 


Dramatic play 
Schedule (creative) 


Fumisking and Display for ChOdren 
Furnishings (routine) 


Supervision (creative) 


Fumi^ngs (learning 


Social Devdopmc.it 


Fumisiung relaxation 


Space (alone) 


Room arrangement 


Free play 


Child related display 


Group time 
Cultural awareness 


I^®g*>*ge- Reasoning Experiences 


Tone 


Understanding language 
Using language 
Reasoning 
Informal language 


Exceptional provisions 

Adult Needs. 


Fine and Gross Motor Activities 


Adult personal area 


Fine motor 


Adult opportunities 


Supervianon (Fine Motor) 


Adult meeting area 


Gross Motor space 
Gross Motor equipment 
Gross Motor time 
Gross Motor supervision 


Parent provisions 



HOW HAS IT BEEN USED ? 

Many investigators now use the "Early Childhood Environment Rating Scale" to 
evaluate the quality of nursery settings. It is being used for regulation and monitoring and 
for staff training, as well as being u^ as a research instrument. 

A number of investigators have found correlations between nursery scores of quality 
on the ECERS and measures of children's language, intellectual and sodal development ( for 
^cample, in terms of vocabulary (Schliecker et al. 1991) and social adjustment (Kontos 1991). 
(Hie scale was used as one of the measures of assessing daycare quality in the High Scope 
Perry project (1993).) 

The m^ority of the studies have been carried out in North America or by American 
investigators, in the USA by Bryant et al 1991, Kontos & Fi«ie 1987, Mednick & Fioie 
1990JFiene & Mednick 1991, Kontos 1991, Howes et al. 1992, Scarr et al. 1994 , Whhdiook, 
Howes & Phillips 1989, and in Bermuda, by US researchers, (McCartney 1984, Phillips, 
McCartney & Scarr 1987), and in Canada, (Goelman & Pence 1986, Goelman & Pence 1987, 
Schlieker et al. 1991). However it is now b«7^nning to be used by researchers in other 
countries. Farquhar (1990) used it in New Zealand, Rossbach (1990) used it in a study of 68 
German Kindergartens and Statham and Brophy (1991) have recently used it in Britain to 
assess the quality of playgroups. 

This use, in contexts outside the US has prompted at least one set of investigators 
Brophy and Statham, to have reservations about its appropriateness for early childhood 
settings winch are different from those in which the scale was developed. 

This paper will explore this issue frirther and consider to what extent the ECERS 
remuns a valid instrument when used to make cross-national comparisons of quality. 

METHOD 

The author used the ECERS as part of a qualitative investigation into nursery 
pro>dsion and practice between 1990 and 1994 in cities in four countries. New Ycrk,(USA), 
Barcdona (Spain), Arezzo (Italy), and Hamosand, (Sweden) . 
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The nurseries in which the author carried out observations and interviews were chosen 
by contacts in the countries concerned for both their willingness to participate and for their 
representativeness. A minimuni of four institutions which took children under three (except 
in Sweden, where one of the nurseries took children from (3-7), were visited for at least half a 
day in each chy. 

RESULTS AND DISCUSSION 
The Major findings 

There was a large amount of variability in the scores achieved by nurseries in New 
York, where in some nurseries items were given scores as low as one or two, considered to be 
inadequate , while in oth^ nurseries itms were scored in the excellent category, (a score of 
seven). The nursery mean scores ranged between 2.6 and 5.4. 

In contrast nurs^es in Harfosand, Sweden uniformly tended to g^ excellent scores. 
In Arezzo scores were usually in the range of 6 or 7 , and in Barcelona there was a little more 
variability but item scores were usually in the good to excellent range of 5-7. (A mean score 
of at least 5 on the ECERS is the score which Bryant et al. (1991 p791) believes is the 
minimum for devdopmentally appropriate practice.) 

The mean scores in the various countries were respectively; 

New York 4.3 
Barcelona 5.4 
Ar^zo 6 
Hamosand 6.9 

As can be seen, the New York nurseries were less likely to achieve 'good' ratings, 
(scores above 5) . 

However these figures need interpretation and qualification for the reasons which this 
p^er will discuss. 

There are three main problems. They are firstly a problem of 'meaning';secondly, the 
standards being too minimal;and thirdly omission. These concern both individual it^ns and 
the scale as a whole. 
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To illustrate these points some examples will be given. 

LMeaning 

There is often a problem of the meaning of the items. When settings are different the meaning 
of the presence or absence of particular materials or of an activity can be different. 

Because scores were dependent on the presence of certain equipment, experiences or 
ways of scheduling that are common in North America,it was not always clear \^diether to 
interpret items literally or in terms of what one believed to be their intended meaning. 

For example, 

(i) Item 24, SandA¥ater 

The notes say " The intent of this item is that children have outidde and inside (if needed 
because of weather conditions) regular access to sand and water.” 

This is difficult to score in Sweden. There, they did not necessarily have sand and 
water inade, but it was always available rnitside. When challenged as to what happened when 
it rained, they smd that the children loved it then, th^ could carry buckets of water fi'om place 
to place, create water^ls down slopes, pour it into the sand. They smd there were no bad 
weather conditions, only bad clothes. (The children in these nurseries were always equipped 
with changes of dothes for the outside; Wellington boots; waterproof coats and ha^s , and 
other equipment like skates, and sledges.) 

(ii) This emphasis on the outdoors also had implications for items in other subscales, 
such as those concerned with fine and gross motor play. In Sweden they had all the materials 
but children seemed to spend less time inside at table top activities than in, for instance, 
Arezzo. Instead , in Sweden, the children showed their fine motor skills in other ways. - in 
putting on and taking off their own outside clothes several times a day by themselves. This 
involved fiistening coat buttons and shoe laces, and other fine motor tasks. (This was in 
contrast to New York, where in one nursery, children had to wiut silently with their hands 
under the table, wdiile they were selected, one by one, to be dressed in thdr outdoor clothes by 
a member of staff.) 

Thus self hdp and fine motor ^lls could be acquired through handlin g one's own 
clothes and bdongings and outdoor equipment as in Sweden , yet the assumption of the items 
concerned with fine motor skills, seemed to be that these would be provided by a variety of 
taUe top activities. 
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2. Standards too minimal 

The standards for scoiing can be too minimal. 

(i) In item 22,* Mu»c', the quality of experience is not scored. The criteria are 
whether muscal experiences are provided , so for an excellent score a nursery must provide 
music daily. The ratings do not indicate whether tho-e is specialist music teaching for the 
children. For exanqsle, it is often wcpected in Sweden that staff have the ability to pl^ the 
guitar or piano. In Barcelona there are visting muse specialists who develop a music 
curriculum fiir children under three. For example, nine month old babies listen to short 
excerpts of Mozart; small groups of toddlers indicate rhythm and pitch, while music is played 
to them. 

This emphasis on developing a musical education, is not picked up by an item for 
which an excellent score can be gained if the childrra's carers have regular sin gin g sessions 
with them. There is no item which asks the observer to check for the musical adequacy of the 
experience, its depth or quality. 

Other examples could be given. 

(ii) The item on cultural awareness has been referred to by other investigators (Brophy 
and Statham) as being insufficient to cover the emphasis on antiracist and multicultural aims 
that the playgroups th^ studied, had. Nor was it sufficient to cover the various ways in 
which related issues were seen as important, e.g in Barcelona the emphasis on a Catalan 
heritage and in a black nursery in Harlem, New York , in giving the children a pride in black 
culture. The item was written in such a way as to presume one was introducing children in a 
m^ority culture to the needs of the minority, not that one was trying to redress the balance for 
'minority* (the less dominant cultures's) children. 

(iii) Tone. 

There was only one item which covets the tone of a nursery. This is not suffident to 
cover the emphasis that is put on cliildren’s care for each other (in Sweden), what they call 
'sibling relationships'. 



3. Omisdon 

Tliere is a prdilem of omission, both the omisrion of items but also ffie omission of 
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whole dimensions of experience. 

(i) In some nurseries there was obvious care for the staffs well being, their backs for 

example. In Sweden this was shown by specially constructed high chairs for the children,and 

in Arezzo and Barcelona by steps for toddlers to climb up to the chan^ng table. In contrast in 
New York, there was little ewdence of thought for stafis' needs. ' 

But there was an omission of any itan that would score for this J 

For example, the criteria for scoring for routine furnishings was that there should be I 

sufficient number ofjwecesofchildazed routine care forniture in good repair." These should 

be "child azed: child's feet should rest on floor when seated in chair; tifole height comfortdrle ftj 

(knees fit under t^le, elbows above table). In Sweden there was child sized furniture. 

However there were also high chairs provided which were specially designed with steps tha t 

the children could n^otiate themselves, so that they still had autonomy, but which were also ■ 

designed with the needs of the staff in nund as well. The chairs protected the backs of the 

staffs by not always requiring them to bend and allowed them to be comfortable at shared 
meal-times with the cluldren. 

There was no item which coded for concern for the stafis' well being. Yet making the 

staff feel that they were cared for and also valued, could well have impacted on their relations 
with the children. 

(ii) But this omission is a pointer to a further lack of a dimension concerning how 
staff are valued and the nature and quality of relationships between staff. 

This is an indication of the omission of other items concerning institutional structure 
or organisation, which for instance might concern staff stability or the organisational system. 

One cannot code for whether staff relationships are hierarchical or cooperative/democratic ;(as 
for example in Barcelona, which is non-hierarchical in organisational structure and where there 
is a rotating elected supervisor.) 

Oil) Nor was there any way ofscoring for an aesthetic dimension. In Arezzo for i 

. . ’ 
instance, it was clear that there was an effort to make the nurseries beautifiil places for the | 

children to be in, and in which to work, with an emphasis placed on the quality of the materials 
used; on light, i^ace and colour. 

(iv) Children's relationdiips wth each other. 

There is a lack of items or of a whole dimension relating to children's relationships with 
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each other. For example, are these planned for and encouraged? In Italy and Spain, children 
are with others the same year of age. They are expected to become members of a group. 
This has other implications for the stability of groupings in terms of both children and staff 
(whether key worker systems operate or not). 

In Sweden children are not only with others the same age but also with children both 
younger and older in what are called sibling groups. Here a further aspect is added, not only 
the encouragement of iiiendships, but also of caring for each other, kindness and taking 
responsabilily. 

(v) The taking of responsibility 

In Sweden this encouragement of the taking of responsibility also extends to taking 
responsibiUty for the environment. ChUdren are encouraged, for example, to look after and 
water plants inside the nursery and to take an interest in, and responsibility for caring for, 
plants,insects and animals, as p^ of the natural environment outside the nursery. Such values 
"democracy, equality, solidarity, security, responsibility" are stated as part of the Swedish 
Government guidelines for nurseries (The National Board of Health and Welfare) ,but not 
reflected in the ECERS. 

One nursery in Sweden described their aims as: 

To ensure that children and parents should feel s^«e and welcome everyday. 

To teach children; 

that each person is valued; i 

to take good care of each other; 
to show kindness; 

and to take responsibility for themselves and the group. 

These are not cognitive aims and they are also ones directed at the present, not just the 
future. ( I did not see a quarrel between children in the Swedish nurseries.) 

Such diflferoit urns and values have a pervasive effect since they will affect the 
organisation of the nursery, the nature of the grouping system, age or sibling grouping and the 
stability or transitoiy nature of staffing or children's Roupings. They may affect assumptions 

about room layout, and how many different rooms, (whether open plan, or classroom based) 
and stafi^ cluldren have access to. 



(vi) Present versus fiiture directed 

But in particular the emphasis on the aesthetic, the emphasis on relationships between 
the duldren in the here and now, and the emphasis on children enjoying themselves, that is in 
the Swedish nursery, all suggest an interest in the present that perh^^)s is lacking in what 
appears to be the achievement, and future directed cognitive bias of many items on the 
ECERS. 

CC-NCLUSIONS 
Advantages and Limitations 
Advantages 

It can be undertaken in two hours. It can provide comparison measures on a number 
of criteria which child care professionals have agreed are significant. 

Limitations 

Its limitations arise from the way in which the scale was developed. The items and criteria for 
scoring were chosen from those thought important by experts in the USA. But the theoretical 
assumptions behind the item choice, are not stated. 

However the notes for scoring seem to fit well with certmn American cognitively 
based, achievement oriented and future based philosophies. There seems to be an implicit 
curriculum which perhaps fits well with Bredekamp's (1986) concept of developmental 
appropriateness. (This is discussed in Bryant et al. 1991.) However because the theoretical 
justification is not made explicit, it is difficult when u^g the measure in countries whidi do 
not share the American, or perhaps, English speaking, p^chologtcal child development 
tra^on, to always know whether nurseries should be ^ven high or low scores when making 
judgements about a numb«’ of the items. Scores may wdi depend on whether the scoring 
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system is followed literally or whether interpretations are made which take cultural and/or 
theoretical differences about curriculum aims and practices into account. 

Where provision is uniformly good and in Hamosand, Sweden, and Arezzo, Italy, 
where there was little variation and almost all the nurseries scored in the good to excellent 
range on most of the criteria, then we may need new measures developing in order to 
differentiate between nurseries and these measures may need to meet the curriculum aims and 
criteria which the staff and organisers of the child care and educational system believe to be 
important. 

But currently many aspects of nurseries, which may affect their quality in terms of the 
environment they provide for children are not picked up by the ECERS and its use can 
obscure rather than illuminate, what different countries see as the most significant aspects of 
the care and education system and curriculum. 

It would always be possible to add further items and miss out others but the meanings, 
goals, purposes, and aims behind the provision of any piece of equipment, activity, schedule or 
organisational structure may be different in different settings. Therefore the theoretical 
justifications and reasons for selection need to be made explicit. 

There is also a wider discussion to be had about the meaning of quality (Pence & Moss 
1994). It may not be only the child's experience but also those of workers and parents, of 
society and of other stakeholders that need to be taken into account. 

Therefore we need to develop measurement tools which explicitly state the values and 
theoretical perspectives behind their construction. 
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ABSTRACT 



Using the Eariy Childhood Environment Rating Scale as a measure to make cross- 
national evaluations of quality: Advantages and Limitations 

How useful is the Early Childhood Environment Rating Scale (ECERS) as a research 
instmment in making cross-national comparisons of the quality of nurseries'’ 

The paper briefly outlines the ECERS and discusses the extent to which it can be used 
as a universal evaluative tool to measure quality. The author refers to her use of the EiCERS 
scale in cross national contexts, to comment on its advantages and its limitations 

The author used the ECERS as part of a qualitative investigation into nursery^ 
provision and practice between 1090 and 1994 in cities in four countries. New York.(USA) 
Hamosand, (Sweden), Arezzo (Italy) and Barcelona (Spain). The nurseries in which the 
author carried out observ-ations and interviews were chosen by contacts in the countries 
concerned for both their willingness to participate and for their representativeness. A 
minimum of four institutions which took children under three were visited for at least half a 
day in each city 

The author uses examples derived from this research to argue that while the scale may 
have some ad\antages in that, ratings can be undertaken in two hours, and the scale can 
provide comparison measures on a number of criteria which child care professionals have 
agreed are significant, it also has a number of disadvantages Because the scale is empirically 
rather than theoretically based and is not explicit about the evaluative categories which 
underlie it, its use can obscure rather than illuminate, what difl'erent countries see as the most 
significant aspects of their care and education provision 

file paper concludes that we need to de\eU>p ineaMirement tools which explicidv state 
the values and theoretical perspectives behind their construction 
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